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Abstract Body 


Background: 

A converging base of empirical support exists for the use of explicit instruction during early 
literacy and beginning mathematics instruction (NMAP, 2008; NRP, 2000). Archer and Hughes 
(2010) define explicit instruction as a structured, instructional approach for teaching foundational 
concepts and skills in key academic areas, such as reading and mathematics. Central to this type 
of instructional approach is the facilitation of explicit teaching episodes. Teaching episodes refer 
to the active mechanisms of instruction hypothesized to increase treatment intensity and, in turn, 
improve student achievement (Warren, Fey & Yoder, 2007). 

In this study, we conceptualize such teaching episodes as an integrated set of observable 
student-teacher interactions. Instructional interactions that take place between teachers and 
students around critical academic content are a defining characteristic of classroom instruction 
and a component carefully defined in many education interventions (Cohen, Raudenbush, & 

Ball, 2003; Pianta & Hamre, 2009). Recent observation research highlights the importance of 
providing frequent instructional interactions to improve student outcomes in early reading 
(Smolkowski & Gunn, 2012) and beginning mathematics (Doabler, Baker et ah, 2013). 

Focus of the Research: 

This presentation will focus on observational data collected during two separate IES-funded 
randomized controlled trials. The first study, Project Enhancing Core Reading Instruction 
(ECRI), is a four-year project designed to study the efficacy of a multi-tiered, first-grade 
intervention system. The second study, the Early Learning in Mathematics (ELM) project, 
investigated the efficacy of a core (Tier 1) kindergarten mathematics curriculum. 

There are two primary purposes of this presentation. The first is to establish classroom 
stability of instructional interactions during early literacy and beginning mathematics instruction. 
The second purpose is to examine the association between these instructional interactions and 
academic achievement. Three research questions were posed to guide our research on 
instructional interactions: (1) How stable are instructional interactions across the academic year? 
(2) How reliable are classroom means of instructional interactions (averaged across 
observations)? (3) Is the rate of instructional interactions associated with student achievement? 
Setting: 

The ECRI study was conducted in 44 public elementary schools from Oregon and 
Massachusetts during the 2009-2010 and 2010-2011 school years. Within district, schools were 
randomly assigned to either the ECRI treatment or a wait-list comparison condition. From these 
44 schools, a total of 138 classrooms were included in the analyses. 

The ELM study was conducted in 46 schools (32 were public schools, 1 1 were private, and 3 
were charter schools) from Oregon and Dallas, Texas during the 2008-2009 and 2009-2010 
school years. Participating schools were located in urban and suburban areas. Among the 46 
schools, 129 classrooms were randomly assigned to either treatment or a control condition. 
Participants: 

The ECRI study sample included 138 first grade classrooms, each taught by certified 
teachers. The majority of teachers were female (97%) and had an average of 13.5 teaching 
experience. Within the 138 classrooms were 3,022 first grade students. Student ethnicity as 
reported by schools was 65.27% White, 21.38% Hispanic, and 2.52% African American. 
Approximately 20% of students were identified as ELs, 46% received free or reduced price 
lunch, and 14% were identified as needing special education services. 
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The ELM study included 129 kindergarten classrooms taught by 130 teachers. One 
classroom had two teachers, each working a half-day schedule. Most teachers were female (98%) 
and had, on average, 5.5 years of teaching experience. In the 129 ELM classrooms were 
approximately 2,200 students. An average of 76% of the student population qualified for free or 
reduced lunch programs. In the Oregon schools, the ethnicity breakdown was Hispanic (36%), 
Black (2%), White (56%), Asian and Pacific Islander (5%), and American Indian (1%). In the 
Texas schools, the ethnicity breakdown was Hispanic (69%), Black (29%), White (1%), Asian 
and Pacific Islander (<1%), and American Indian (<1%). 

Interventions: 

ECRI Tier I intervention. ECRI treatment classrooms provided at least 90 minutes of 
daily Tier I, whole group, reading instruction, using a district-adopted core program. Lesson 
maps and instruction templates were used to enhance the content, design and delivery of 
instruction in ECRI treatment classrooms. To promote reading proficiency for all students, ECRI 
lesson maps and instruction templates emphasize the following features of explicit instruction: 

(a) learning strategies are made conspicuous; (b) instruction builds connections between new and 
previously taught content; (c) instruction activates student background knowledge; (d) 
instructional scaffolds are provided to promote success; (e) sufficient practice to achieve 
automaticity is provided; and (f) previously learned material is reviewed systematically. In 
addition, lesson maps , constructed for each core program, prioritize for teachers the essential 
components of reading instruction; these maps were designed to ensure that instructional time is 
spent teaching essential content. Instruction templates were designed to increase the intensity of 
instructional interactions by incorporating clear teacher models, ample guided and structured 
practice opportunities for students, and consistent teacher feedback. ECRI teachers received five, 
eight-hour days of professional development (PD) across the academic year. Expert coaches also 
provided in-class coaching support to treatment teachers as an additional means of PD. 

Comparison schools in the ECRI study used a comprehensive core program identified 
and adopted by standard district procedures for 90 minutes each day during Tier I instruction. 
Within each district, the core program used in comparison schools was the same program used in 
ECRI treatment schools. However, comparison teachers did not receive enhanced ECRI 
materials (i.e., lesson maps and instruction templates). Comparison teachers received typical 
district-approved PD activities. 

ELM intervention. The ELM curriculum is a full-year kindergarten mathematics 
curriculum designed for use in whole classroom settings. It functions as a core (or Tier I) 
mathematics program and has a primary aim of promoting students’ development of early 
mathematics proficiency. ELM includes 120 core daily lessons that are approximately 45 
minutes in duration. Lessons incorporate 4-5 activities across four content strands: (a) whole 
number and operations; (b) measurement; (c) geometry; and (d) precise mathematics vocabulary. 
Mathematics content is explicitly introduced in each lesson, and systematically reviewed and 
extended across lessons. ELM teachers are expected to model and demonstrate what they want 
students to leam, and provide specific and frequent feedback to students during the learning 
activities. Teachers are also expected to provide students with frequent opportunities to practice 
key mathematics concepts, such as opportunities for students to verbalize their mathematical 
thinking and understanding (Gersten et al., 2009). ELM teachers received four days of PD 
training related to program implementation (i.e., every 40 ELM lessons). 

Mathematics instruction in the control condition consisted of standard district mathematics 
practices, and included the use of teacher-developed materials and a number of different 
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published curricula, such as Everyday Mathematics and Bridges in Mathematics. The 
instructional focus varied, with some teachers emphasizing whole number concepts, and others 
focusing on particular aspects of geometry and measurement. Instruction was delivered through a 
variety of different mediums, including whole-class and small group activities. 

Significance of study: 

The ELM and ECRI efficacy trials are among the only rigorous studies that have examined 
closely the association between explicit instructional interactions and student outcomes. Our 
focus on instructional interactions goes well beyond surface measures of implementation fidelity 
by deeply targeting the nature and function of classroom instruction to increase student 
achievement. To our knowledge no studies have conducted a concurrent investigation of explicit 
instructional interactions that occur during both early reading and beginning mathematics 
instruction. This study therefore fills a void in the knowledge base by cogently examining the 
role of explicit instructional interactions across these key academic fields. 

Research Design: 

The ECRI study employed a cluster-randomized controlled trial research design, nesting 
students and first grade teachers within schools. Within district, schools were randomly assigned, 
blocking on district, to either the ECRI treatment or a wait-list comparison condition. The ELM 
study used a randomized controlled design, nesting students within kindergarten classrooms. 
Blocking on schools, kindergarten classrooms were randomly assigned to either the ELM 
program or a control condition. 

It is important to note that the present study does not conduct tests of experimental condition, 
but rather focuses on examining the relationship between explicit instructional interactions and 
student achievement. The primary unit of analysis for the present study is the classroom. 

Data Collection and Analysis: 

Trained staff members collected all student assessment and classroom observation data in 
both projects. Data collection met acceptable reliability criteria for ah student and classroom 
measures included in the analysis. For both projects: (a) student performance measures were 
collected for all students at pretest and posttest and (b) observation data were collected in all 
classrooms, regardless of condition, three times during each study year. 

Student measures. In ECRI, student reading measures included the Stanford 
Achievement Test, Tenth Edition (SAT-10), and DIBELS (6th Edition) Nonsense Word Fluency 
(NWF) and Oral Reading Fluency (ORF). In ELM, student math measures included the Test of 
Early Mathematics Ability (TEMA-3rd Edition) and the Early Numeracy Curriculum-Based 
Assessments (EN-CBM). All student measures demonstrate strong psychometric properties. 

Classroom Observations of Student Teacher Interactions (COSTI). The ELM and 
ECRI projects used a modified version of the COSTI instrument (Smolkowski & Gunn, 2012) to 
document instructional interactions. The modified COSTI measures the frequency of explicit 
instructional interactions by specifically targeting six instructional interaction behaviors: two at 
the teacher level and four at the student level. All behaviors are coded in a continual, serial 
fashion and thus each behavioral occurrence is recorded. For teacher behaviors, observers coded 
teacher demonstrations and academic feedback. Student behaviors included: (a) group responses, 
(b) individual responses, (c) errors, and (d) other forms of student practice. All classroom 
observations were scheduled in advance and conducted during core instruction, with observers 
remaining in each classroom for the duration of the instructional period. Observations were not 
scheduled according to specific math or reading content, or a particular instructional day. 
Analysis: 
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Eight instructional variables were investigated in the study, including five sets of 
instructional interactions, the duration of instruction, and two conditional probabilities (see Table 
1). We measured the stability of variables across time and the reliability of classroom means by 
calculating intraclass correlation coefficients (ICC) from three observations nested within each 
of the participating ELM and ECRI classrooms. The stability ICC represents the proportion of 
between-classroom variance, whereas the within-classroom variance provides an estimate of the 
day-to-day variability in instructional activities plus any unreliability in the measure. Higher 
ICCs suggest fewer observations are necessary to obtain a reasonable estimate of behavior across 
the year; whereas lower ICCs indicate unstable behavior and imply that more observations are 
needed to better capture the behaviors of interest. 

We analyzed the association between the eight instructional variables and gains in student 
achievement by nesting students within classrooms. For the ELM study, approximately 2,220 
students were nested in 129 classrooms, whereas in the ECRI study, approximately 2,954 
students were nested in 138 classrooms. To ease the interpretation of results, we computed effect 
sizes for each instructional variable as the partial correlation with each student outcome 
measured in the spring, after controlling for pretest achievement levels. Partial r values represent 
the classroom-level effects of the observation measure on each outcome. 

Findings: 

Table 2 displays descriptive statistics and ICCs for each of the instructional variables. The 
ICCs in Table 2 indicate that the instructional variables were not stable across the academic year 
within classrooms. For example, 29% of the variability in rates of the first set of interactions 
occurred between classrooms, leaving 7 1 percent of the variability within classrooms. The 
reliability of classroom means, also depicted in Table 2, indicates that many of the instructional 
variables fell below a threshold of .70. Exceptions were duration of mathematics instruction (.91) 
and the ratio of group/individual responses to other forms of students practice during reading 
instruction (.76). A reliability of the mean greater than .70 is necessary to compute an accurate 
estimate of true classroom practices (Smith, McCarthy & Anderson, 2000). 

We tested the associations between these instructional variables and gains in academic 
achievement within a series of multilevel models in which students were nested within 
classrooms. Table 3 provides results that indicate small trend-level associations in the ECRI data 
were obtained for the fourth instructional variable (rate of group or individual response 
sequences). These positive effects were found on DIBELS ORF and NWF (Words Recoded 
Completely and Correctly). Analysis of the ELM data indicates that a small, yet significant, 
effect was obtained between duration of observation and gains in EN-CBM scores (see Table 3). 
Conclusion: 

Standardized observation protocols have demonstrated the capacity to measure core elements 
of intervention trials, including implementation fidelity and intervention impact (Snyder et ah, 
2006). In this study, we used a direct observation tool to systematically investigate the explicit 
instructional interactions that occur during early literacy and beginning mathematics instruction. 
Preliminary analysis of the ELM and ECRI data reveals that many of the studied instructional 
variables were unrelated to student achievement. One explanation for these results is that the low 
stability of the instructional variables attenuated the relationships. The present study highlights 
the importance of obtaining stability of observed teaching practices (Shavelson & Dempsey, 
1976). The implications of the present study warrant further investigation of methods to improve 
stability of these instructional variables, such as controlling for the type of academic content 
observed and observing consecutive days of instruction. 
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Appendix B. Tables and Figures 


Table 1. Instructional Variables Explored in Projects ECRI and ELM 

Variable Instructional Interaction Behaviors 

1 Rate of model or feedback; (< 5 consecutive student responses 
[individual/group/other forms of student practice] between teacher behaviors); 
model or academic feedback 

2 Rate of individual or group response; error; academic feedback 

3 Rate of any number of student responses; model or academic feedback 

4 Rate of group responses; individual responses 

5 Rate of all behaviors 

6 Ratio of individual to group responses 

7 Ratio of individual or group responses to other forms of student practice 

8 Duration of instruction 
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Table 2 


Descriptive Statistics and Stability ICCs for Instructional Variables Observed in Projects ELM 
and ECR1 


Instructional 

Variable 



ECRI 




ELM 


M 

SD 

ICC 

Reliability 
of the Mean 

M 

SD 

ICC 

Reliability 
of the Mean 

Variable 1 

0.8 

0.5 

.39 

.56 

0.8 

0.4 

.29 

.55 

Variable 2 

0.1 

0.1 

.26 

.42 

0.1 

0.1 

.18 

.40 

Variable 3 

0.9 

0.5 

.40 

.57 

0.9 

0.4 

.32 

.59 

Variable 4 

0.1 

0.1 

.46 

.63 

0.1 

0.1 

.10 

.24 

Variable 5 

4.8 

2.5 

.36 

.53 

3.3 

1.4 

.11 

.26 

Variable 6 

3.8 

13.3 

.12 

.21 

1.0 

1.2 

.19 

.41 

Variable 7 

29.0 

27.8 

.61 

.76 

10.9 

11.9 

.36 

.62 

Variable 8 

84.2 

24.0 

.46 

.63 

38.3 

10.5 

.77 

.91 


Note. M = Mean across all observation occurrences and teachers. SD = Standard Deviation. ICC 
= intraclass correlation coefficients. 
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Table 3 


Associations Between Instructional Variables and Reading Outcomes in Projects ECRI and 
Mathematics Outcomes in Project ELM 


Instructional Variable ECRI ELM 



SAT- 10 

ORF 

CLS 

WRC 

TEMA-3 

EN-CBM 

Variable 1 

-.02 

.11 

.10 

.14 

.09 

-.01 

Variable 2 

-.10 

-.11 

-.16 

-.17 

-.11 

-.01 

Variable 3 

-.01 

.12 

.12 

.17 

.08 

<.01 

Variable 4 

-.06 

.20- 

.09 

.20- 

.11 

.10 

Variable 5 

.04 

.14 

.13 

.18 

.02 

<.01 

Variable 6 

.10 

-.01 

.06 

.06 

.09 

.10 

Variable 7 

-.03 

-.02 

-.02 

.02 

-.08 

-.09 

Variable 8 

.06 

-.05 

-.07 

-.07 

.09 

24 ** 


Note. Table reports partial r values for the classroom-level effects of the observation measure on 
each outcome controlling for student-level pretest scores. ORF = DIBELS Oral Reading Fluency. 
CLS = DIBELS Correct Letter Sounds of Nonsense Word Fluency. WRC = Words Recoded 
Completely and Correctly of Nonsense Word Fluency 

~p < .10, *p < .05, **p < .01 
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